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ABSTRACT 

In this paper, we introduce a new iterative method which we 
call one step back approach: the main idea is to anticipate 
the consequence of the iterative computation per coordinate 
and to optimize on the choice of the sequence of the coordi- 
nates on which the iterative update computations are done. 
The method requires the increase of the size of the state 
vectors and one iteration step loss from the initial vector. 
We illustrate the approach in linear and non linear iterative 
equations. 

Categories and Subject Descriptors 

G.l.O [Mathematics of Computing]: Numerical Analy- 
sis — Numerical algorithms; G.1.4 [Mathematics of Com- 
puting]: Numerical Analysis — Iterative methods 

General Terms 

Algorithms, Performance 
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Numerical computation, iterative method, fixed point 

1. INTRODUCTION 

Iterative methods to solve large sparse systems have been 
gaining interests in very different research areas and a large 
number of approaches have been studied: starting from Ja- 
cobi or Gauss-Seidel iteration methods, more recent works 
focuses on relaxation methods, Krylov subspace methods, 
the use of preconditioners, matrix decomposition, parallel 
computation etc. to solve linear or non linear problems [14| . 
El, DP, El, [S], El, El, Uni to cite few of them. 

In this paper, we propose a new iterative algorithm based 
on the anticipated consequence of the iteration at each co- 
ordinate level: we believe that this approach may bring sig- 
nificant improvement in a large class of linear and non linear 
problems and it seems that such an approach have not yet 
been studied by the research community. More precisely, we 
study the computation of the fixed point X £ IR^ solving 

X = H(X) 

starting from the initial condition Xq G IR^. 

In linear algebra, this approach is equivalent to the idea 
of fluid diffusion that was called D-iteration in [Tlj . 

In this paper, we will not consider the convergence issue 
and assume that the Jacobi or the asynchronous version of 



Gauss-Seidel style iteration of = H(X„) converges to 

a unique fixed point (that's basically contracting operators, 
cf. for instance [111110). 

In Section [2l we define the notations and the proposed 
One Step Back (OSB) algorithm. Section El illustrates two 
concrete examples to solve linear and non linear systems. 

2. OSB ALGORITHM 

We will use the following notations: 

• I G IR'^''^ the identity matrix; 

• Ji the matrix with all entries equal to zero except for 
the i-th diagonal term: {Ji)ii — 1; 

• H : IR'^ -s> IR'^ an operator; 

• X = {ii, i„, ■■■} a sequence of the coordinates (or 
nodes), i„ G {1, .., A^}. 

2.1 Iterative equations 

The OSB iteration is defined by the triplet (H, Xo,T) and 
exploits two state vectors of size N, H„ (history) and Fn 
(residual fluid) based on the following iterative equations: 

Ho = Xo 

Hn = i?n-l+J.„(Fn-l) (1) 

and 

Fo = H(Ho)-i/o 

F„ = (I- J.J(F„_i) + H(i/„)-H(i/„_i). (2) 

2.2 Sequence choice and distributed compu- 
tation 

The choice of the sequence T depends on the computation 
costs structure that need to be optimized. One may consider 
those introduced in [T2], [T31 or [S] depending on the context. 

For the distributed computation architecture, one may 
also use the one proposed in [10] for the linear case: the 
architecture for linear or non linear cases should be a priori 
the same and benefits from the asynchronous properties of 
the proposed method. 

3. ILLUSTRATION 

3.1 Linear equation: D-iteration 

If we take H = P a linear operator, we have: 

F„ = (I- J,J(F„_i)-f H(iy„)-H(//„_i) 

= (I — Ji„)-Fn_l -|- PJi„_F„_l. 



Then we find back D-iteration equation cf. [TT]. The com- 
putation gain for this case has been studied in details for 
instance in |12l 1101 113) and is skipped here. 

3.2 Non linear equation: simple example 

Consider the non-linear fixed point problem of dimension 
three: 

H(x, y, z) = +l,{x + z)l^ -fl, (x + y)/4). 

The fixed point of the above equation is ((23-1-^379) /lO, (23+ 
\/379)/30-hl6/15, (23-hV379)/30-f4/15) ~ (4.247,2.482, 1.682). 
The updates of the fiuid vector F„ is based on the three in- 
crement functions: 

(a-' — x) (x' — x) , 
4 ' 4 ' 

{y - y) \ 



H(2;',y,z) - H(a;,i/,z) = {^(V^' - ^fi) 
H(2;,y',z) -H(a;,y, 2) = (\/^(i/y^ - ^y). 
H(x,y,2')-H(:c,y,2) = (0,^^^^,0). 



Figure [T] shows the convergence of Jacobi, Gauss-Seidel 
and OSB iterations. One iteration is here defined as an 
update of three entries (with multiplicity for OSB) of the 
iterated vector. For OSB, we choose the n-th coordinate 
equal to the argmax of Fn in absolute value. The initial 
condition is set to (4.2, 1, 1.5). 
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Figure 1: Illustration: A*" = 3. 

Figure [1] illustrates well the advantage of optimizing the 
coordinate sequence order when the initial vector has some 
coordinates more closer to the limit than others: the gain 
factor compared to Jacobi iteration is here of two orders of 
magnitude at iteration 10! Note that in this case, starting 
from (0, 0, 0), there is no big differences between those three 
methods (the matrix size is too small to observe significant 
differences). 

This example is only shown for the sole purpose of the 
illustration of the potential impact and no theoretical guar- 
antee on the convergence gain is given here. However, the 
author believes that we should have cases where the OSB 
method may provide substantial convergence improvement, 
since it can be applied to a very wide range of fixed point 
problems. The first factor to be considered to determine 
whether OSB method can improve the iterative computa- 
tion cost is the complexity of the computation of the incre- 
ment H(_H'„) — H(_ff„_i), which may simplify or introduce a 
cost overhead to Gauss-Seidel style normal iteration, and to 



compare this complexity to the gain brought by coordinate 
level optimization. 

Note finally that the one step iteration computation lost at 
the first iteration is recovered at the end considering Hn + F„ 
as the estimator (instead of Hn). 

4. CONCLUSION 

In this paper, we described a new iterative method and 
illustrated its applications to two simple fixed point prob- 
lems. 
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